Smart data storage tiers for data object transitioning

ABSTRACT

An object-based data storage service receives a request to store a data object in association with a smart data storage tier. Based at least in part on characteristics of the data object, the object-based data storage service identifies and stores the data object in a first location corresponding to a first data storage tier. The object-based data storage service monitors access to the data object to identify a second set of characteristics of the data object. This second set of characteristics is used to determine that the data object is to be transitioned to a second data storage tier. The object-based data storage service, based at least in part on this determination, stores the data object in a second location corresponding to the second data storage tier.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/797,372, filed Feb. 21, 2020, entitled “SMART DATA STORAGE TIERS FORDATA OBJECT TRANSITIONING,” which is a continuation of U.S. patentapplication Ser. No. 15/933,242, filed Mar. 22, 2018, entitled “SMARTDATA STORAGE TIERS FOR DATA OBJECT TRANSITIONING,” the disclosures ofwhich are incorporated herein by reference in their entirety. Thisapplication also incorporates by reference for all purposes the fulldisclosure of U.S. patent application Ser. No. 15/933,216, filed Mar.22, 2018, entitled “AUTOMATED TIER-BASED TRANSITIONING FOR DATAOBJECTS.”

BACKGROUND

The use of remote and network-based storage services, such asobject-based data storage services, has proliferated in recent years.Object-based data storage services enable users ranging from largeorganizations to individuals to utilize storage resources provided bythese services for retaining and managing their data. With object-baseddata storage services, users may forego the initial setup costsassociated with purchasing storage equipment, such as hard drives, solidstate drives, and the like. Instead, users may leverage readilyavailable storage resources of a service provider at a fraction of thecost associated with establishing dedicated storage for their data. Asusers access their data over time, they may transition their data todifferent storage tiers to further reduce their cost in storing theirdata. However, manually transitioning data can be cumbersome and resultin inefficiencies that can instead result in added costs to these usersif the rate of access for their data suddenly changes, negating any costbenefit that may be had by using these storage services.

BRIEF DESCRIPTION OF THE DRAWINGS

Various techniques will be described with reference to the drawings, inwhich:

FIG. 1 shows an illustrative example of a system in which variousembodiments can be implemented;

FIG. 2 shows an illustrative example of a system in which an accessevaluation engine of an object-based data storage service transitionsdata objects among data storage tiers and an archival data storageservice in accordance with at least one embodiment;

FIG. 3 shows an illustrative example of a system in which an interfaceis provided to a customer to request automatic transitioning of dataobjects within a logical data container in accordance with at least oneembodiment;

FIG. 4 shows an illustrative example of a system in which an objectplacement engine of an object-based data storage service determinesplacement of a data object from a smart data storage tier in accordancewith at least one embodiment;

FIG. 5 shows an illustrative example of a system in which an interfaceis provided to a customer to request creation of a logical datacontainer in a smart data storage tier to enable optimized placement ofdata objects of the logical data container in accordance with at leastone embodiment;

FIG. 6 shows an illustrative example of a system in which historicalaccess data for data objects stored in a smart data storage tier isutilized to determine a placement for the data objects in accordancewith at least one embodiment;

FIG. 7 shows an illustrative example of a process for monitoring usageof a data object to determine optimal placement of the data object inresponse to a request from a customer to automatically transition thedata object in accordance with at least one embodiment;

FIG. 8 shows an illustrative example of a process for utilizing usagedata for other data objects in a smart data storage tier to determineinitial placement of a data object in response to a request to store thedata object in the smart data storage tier in accordance with at leastone embodiment;

FIG. 9 shows an illustrative example of a process for utilizingcharacteristics and access patterns for an existing data object in asmart data storage tier to determine an optimal storage location for thedata object in accordance with at least one embodiment; and

FIG. 10 illustrates a system in which various embodiments can beimplemented.

DETAILED DESCRIPTION

Techniques described and suggested herein relate to the automatedtransition of data objects among various data storage tiers of anobject-based data storage service based on access patterns andcharacteristics of these data objects. In an example, a customer of anobject-based data storage service submits a request to the object-baseddata storage service to store a data object in a logical data containerthat is classified as being in a standard data storage tier. Thestandard data storage tier may serve as a general purpose storage classfor data objects that are accessed frequently by users. Further, logicaldata containers classified as being in the standard data storage tiermay be designed to provide high throughput, with low latency, to enableimmediate access to data objects stored in these logical datacontainers. In an example, the object-based data storage serviceprovides users with an option to automatically transition data objectsmaintained in the standard data storage tier to an infrequent accessdata storage tier or to an archival data storage service based on usagedata for these data objects.

In one example, if a user opts to enable automatic transitioning of dataobjects to other data storage tiers and/or to the archival data storageservice based on usage data for these data objects, the object-baseddata storage service monitors access to these data objects over time todetermine whether the access patterns for these data objects isconsistent with the data storage tier the data objects are placed in.For example, if the object-based data storage service determines, basedon usage data obtained for a particular data object or group of dataobjects (e.g., arbitrarily grouped by the user of by the object-baseddata storage service, etc.), that the data object is being usedinfrequently, the object-based data storage service may transition thedata object from the standard data storage tier to an infrequent accessdata storage tier. This infrequent access data storage tier may includelogical data containers that enable long-term storage of data objects ata reduced cost. However, users may incur a fee each time the user accessa data object from a logical data container in the infrequent accessdata storage tier. Thus, based on usage data for a data object in theinfrequent access data storage tier, if there is an increase in thefrequency of use for the data object, the object-based data storageservice may transition the data object to the standard data storage tierto enable access to the data object without incurring a fee to accessthe data object.

In an example, to further reduce the cost of storage of data objects, ifthe object-based data storage service determines that a data object hasnot been used over an extended period of time, the object-based datastorage service transitions the data object to an archival data storageservice. The archival data storage service may provide access to dataobjects stored therein in response to a retrieval task submitted to thearchival data storage service. To reduce the cost of storage, theseretrieval tasks may be completed over a period of time and, thus, maynot be instantaneous. In an example, if the object-based data storageservice determines, based on usage data for data objects stored in thearchival data storage service, that these data objects are beingutilized at a rate that makes using the archival data storage serviceinefficient for users, the object-based data storage service transitionsthese data objects from the archival data storage service to a logicaldata container in the infrequent access data storage tier or to alogical data container in the standard data storage tier.

In an example, the object-based data storage service provides users withan option to store data objects in a smart data storage tier. If a useropts to place a data object in the smart data storage tier, theobject-based data storage tier may obtain usage data for the user'sother data objects that are classified as being in the smart datastorage tier, as well as in other data storage tiers (e.g., standard andinfrequent access) and the archival data storage service. Theobject-based data storage service may identify the characteristics ofthe data object added to the smart data storage tier and compare thesecharacteristics to those of other data objects of the user to identifysimilar data objects. Thus, the object-based data storage service mayuse the usage data for each of these similar data objects to identify aninitial placement for the data object added by the user. This mayinclude placement of the data object in the standard data storage tier,the infrequent access data storage tier, or the archival data storageservice. Thus, by adding the data object to the smart data storage tier,the user may avoid having to determine which storage tier to place thedata object initially. The object-based data storage service may monitorthe characteristics of the data object classified as being in the smartdata storage tier and the usage data for the data object to determinethe optimal placement of the data object.

In this manner, an object-based data storage service can automaticallytransition data objects of a user among different data storage tiersbased on usage data for these data objects and without need for users todefine lifecycle policies to enable these transitions. In addition, thetechniques described and suggested herein facilitate additionaltechnical advantages. For instance, because the object-based datastorage service automatically transitions data objects among differentdata storage tiers based on usage data for these data objects, theobject-based data storage service may reduce the cost for users tomaintain their data objects using the object-based data storage serviceand/or the archival data storage service. Further, because theobject-based data storage service determines optimal placement for dataobjects based on usage data for these data objects, the object-baseddata storage service may increase the efficient of its storage systemsas less-frequently accessed data objects are transitioned, over time, tostorage systems in an infrequent access data storage tier or to anarchival data storage service, which may maintain storage systemsconfigured for less frequent access of these data objects.

In the preceding and following description, various techniques aredescribed. For purposes of explanation, specific configurations anddetails are set forth in order to provide a thorough understanding ofpossible ways of implementing the techniques. However, it will also beapparent that the techniques described below may be practiced indifferent configurations without the specific details. Furthermore,well-known features may be omitted or simplified to avoid obscuring thetechniques being described.

FIG. 1 shows an illustrative example of a system 100 in which variousembodiments can be implemented. In the system 100, a customer 102 of anobject-based data storage service 104 submits a request to store a dataobject 114 in a logical data container classified as being in a standarddata storage tier 106. The object-based data storage service 104 may bea service provided by a computing resource service provider. Theobject-based data storage service 104 may be implemented on a computersystem, or abstraction thereof (such as one or more virtual machinesoperating via a hypervisor), implemented using hardware and software,and may comprise one or more processors and memory that storesexecutable instructions whose execution by the one or more processorscauses the computer system to perform operations described herein. Thedata stored in the object-based data storage service 104 may beorganized into data objects, such as data object 114. The data objectsmay have arbitrary sizes and may, in some instances, have constraints onsize. Thus, the object-based data storage service 104 may store numerousdata objects of varying sizes. The object-based data storage service 104may operate as a key value store that associates data objects withidentifiers of the data objects which may be used by the customer 102 toretrieve or perform other operations in connection with the data objectsstored by the object-based data storage service 104. Access to theobject-based data storage service 104 may be through applicationprogramming interface (API) calls to the service or via an interface,such as a graphical user interface (GUI).

In an embodiment, the object-based data storage service 104 providesdifferent data storage tiers for different data object use cases. Thesedifferent data storage tiers may provide benefits to the customer 102and other users based at least in part on the usage of data objectsclassified accordingly. For example, as illustrated in FIG. 1 , theobject-based data storage service 104 provides a standard data storagetier 106, which can be utilized to classify data objects that are to beaccessed frequently by the customer 102 and other users (e.g., datautilized for dynamic websites, content distribution, mobile and gamingapplications, analytics, etc.). Data objects classified as being in thestandard data storage tier 106 may be stored on computer systems thatenable rapid availability with minimal latency. Thus, in some instances,the object-based data storage service 104 may provide cost savings tothe customer 102 by enabling frequently used data objects classified asbeing in the standard data storage tier 106 to be accessed withoutincurrence of a fee per data access.

The object-based data storage service 104 may also provide an infrequentaccess data storage tier 108, which may be utilized to classify dataobjects that are accessed at a lesser frequency than other data objectsclassified as being in the standard data storage tier 106. For instance,a customer 102 may request storage of the data object 114 in a logicaldata container classified as being in the infrequent access data storagetier 106 if the data object 114 is to be used for, among other things,disaster recovery or as a redundant backup of other data objects. Dataobjects classified in the infrequent data storage access tier 106 may bestored on computer systems that enable rapid availability with minimallatency but at a lower cost than that of the standard data storage tier104. However, each request to access a data object classified as beingin the infrequent access data storage tier 106 may be subject to a fee.Thus, if a data object 114 is utilized at a low rate, it may be morecost-effective for the data object 114 to be classified as being in theinfrequent access data storage tier 106, as the savings resulting fromthe reduced cost of storage may outweigh the expense of accessing thedata object 114 at the low rate. It should be noted that while thestandard data storage tier 106 and the infrequent access data storagetier 108 are used throughout the present disclosure for the purpose ofillustration, additional and/or alternative data storage tiers may beutilized for the management of data objects by the object-based datastorage service 104. For instance, as described in greater detail below,data objects may be classified as being in a smart data storage tier,which may result in monitoring of these data objects to determine anoptimal storage location for these data objects.

In an embodiment, the customer 102, via an interface provided by theobject-based data storage service 104, submits a request to store a dataobject 114 in a logical data container. The request may specify adesired classification for the data object 114, such as a classificationcorresponding to the standard data storage tier 106 or to the infrequentaccess data storage tier 108. Further, in an embodiment, the requestspecifies whether the object-based data storage service 104 is tomonitor access to the data object 114 to determine whether to transitionthe data object 114 from a logical data container corresponding to theselected data storage tier to a logical data container corresponding toanother data storage tier or to an archival data storage service 110.The archival data storage service 110 may be a service provided by acomputing resource service provider or other service provider that maybe accessed and utilized by the object-based data storage service 104 onbehalf of the customer 102.

The archival data storage service 110 may be implemented on a computersystem, or abstraction thereof (such as one or more virtual machinesoperating via a hypervisor), implemented using hardware and software,and may comprise one or more processors and memory that storesexecutable instructions whose execution by the one or more processorscauses the computer system to perform operations described herein. Thearchival data storage service 110 may provide storage for data archivingand backup of customer data, such as the data object 114. The archivaldata storage service 110 may thus persistently store data that may beinfrequently accessed and for which long retrieval times are acceptableto a customer 102 utilizing the service. A customer 102 or theobject-based data storage service 104 may interact with the archivaldata storage service 110 to generate one or more archives. Each archivemay represent one or more data files that may be combined to form thearchive. Accordingly, a customer 102 or the object-based data storageservice 104, through API calls to the service, may upload and retrievearchives from the archival data storage service 110 and monitor theretrieval of these archives, as each retrieval job may typically requireseveral hours to complete.

In an embodiment, in response to the request from the customer 102 tostore a data object 114 in a logical data container corresponding to aparticular data storage tier, the object-based data storage service 104identifies a logical data container that is classified as being in thespecified data storage tier and stores the data object 114 therein. Alogical data container is a logical unit of storage maintained by theobject-based data storage service 104 and may correspond to a unit ofstorage of a computer system, such as a portion of a physical harddrive, solid state drive, memory (e.g., flash memory, random accessmemory (RAM), etc.), a removable drive (e.g., Universal Serial Bus (USB)flash memory drives, floppy disks, external hard drives, etc.), and thelike. Each logical data container may include metadata, which mayspecify the classification of the logical data container (e.g., whichdata storage tier it is a part of, etc.) and the data objects storedtherein. Thus, based at least in part on the data storage tier specifiedby the customer 102, the object-based data storage service 104 mayidentify a logical data container that, according to its metadata, isclassified as being in the data storage tier specified by the customer102.

If the customer 102 has indicated, in its request, that access and usageof the data object 114 is to be monitored to determine an optimalplacement of the data object 114 over time, the object-based datastorage service 104 may monitor the data object 114. For instance, usageof the data object 114 may be recorded in a data log, which may specifythe time at which the data object 114 was accessed and the duration ofsuch access. The object-based data storage service 104 may evaluate thisdata log to identify a frequency over time that the data object 114 isutilized by the customer 102 and other authorized users. In someinstances, the object-based data storage service 104 may compare thisusage frequency of the data object 114 to the usage frequency of otherdata objects classified as being in the same data storage tier. If theusage frequency of the data object 114 is lower than the average usagefrequency for other data objects classified as being in the same datastorage tier, the object-based data storage service 104 may transitionthe data object 114 to a logical data container corresponding to anotherdata storage tier. For example, if the data object 114 is classified asbeing in the standard data storage tier 106, the object-based datastorage service 104 may transfer the data object 114 to a logical datacontainer classified as being in the infrequent access data storage tier108 or to the archival data storage service 110. Similarly, if the usagefrequency of the data object 114 is greater than the average usagefrequency for other data objects classified as being in the same datastorage tier, the object-based data storage service 104 may transitionthe data object 114 to a logical data container corresponding to anotherdata storage tier if the data object 114 is not in the optimal datastorage tier (e.g., standard data storage tier 106).

In an embodiment, the object-based data storage service 104 utilizesmachine learning techniques, such as supervised learning techniques todetermine optimal placement of the data object 114 over time and tocreate prediction metadata, which may specify an optimal data storagetier for the data object 114. A machine learning algorithm may utilize,as input, usage data for a particular data object garnered from one ormore data usage logs for the data object 114. Further, in someinstances, the machine learning algorithm may also utilize usage datafor other data objects of the customer 102 and/or of other similar dataobjects of the customer 102 or of other customers (e.g., same type,stored in a logical data container corresponding to the same datastorage tier of the data object 114, etc.). In some examples, themachine learning algorithm may utilize prediction metadata for othersimilar data objects as input. Other input to the machine learningalgorithm may include a customer's history in selecting the primary datastorage tier for its data objects, a customer's analyzed behavior basedat least in part on the costs incurred to the customer 102 with regardto storage of its data objects, and the like. A machine learningalgorithm may, at any time, utilize one or more sample vectors toperform one or more simulations to determine whether the functionsutilized by the object-based data storage service 104 to determineplacement of monitored data objects are producing correct and accurateresults and/or to refine the one or more functions utilized by theobject-based data storage service 104 to produce correct and accurateresults. For instance, during initialization of the machine learningalgorithm, the object-based data storage service 104 may provide themachine learning algorithm with one or more sample vectors andanalytical results (e.g., desired outcomes) that should be obtainedbased at least in part on these one or more sample vectors. The machinelearning algorithm, based at least in part on this exercise, may adjustthe functions utilized by the object-based data storage service 104 toanalyze the vectors corresponding to activity associated with use of themonitored data objects.

The machine learning algorithm may receive input from one or moreanalysts employed by the a computing resource service provider toanalyze the results from the one or more analyses performed byobject-based data storage service 104 through use of the one or morefunctions described above. For instance, an analyst may review the datalogs and the one or more vectors generated by the object-based datastorage service 104 to determine whether a data object 114 should betransitioned from its current data storage tier to another data storagetier or to the archival data storage service 110. The analyst mayprovide his/her input for use in refining a function used to classifyvector input as corresponding to a decision to transition the dataobject 114 to another data storage tier, to transition the data objectto the archival data storage service 110, or to maintain the data object114 in the current data storage tier. The vector of measurementscorresponding to the review performed by the analyst and the desiredoutcome corresponding to the analyst's input may be used by the machinelearning algorithm to update the function used to classify vectorinputs. Such may be performed by multiple analysts and/or using multiplevector inputs to provide the machine learning algorithm a sufficientnumber of sample vector inputs and desired outputs. The machine learningalgorithm may adjust the one or more functions used by the object-baseddata storage service 104 to increase the likelihood that the desiredresult is obtained in future analyses.

The function used to classify measurement vectors may vary in accordancewith various embodiments. For example, in some embodiments, supportvector machine techniques are used to classify regions in Euclideanspace as indicative of a need to transition a data object 114 to anotherdata storage tier, indicative of a need to transition a data object 114to an archival data storage service 110, or indicative of the dataobject 114 being currently stored in an optimal location. This may beused so that measurements are classified in accordance with the regionin which the measurement vectors fall. In yet another embodiment, themachine learning algorithm can utilize decision tree learning todetermine a decision (classification, regression) tree used to classifyvector input as being indicative of a need to transition a data object114 to another data storage tier, indicative of a need to transition adata object 114 to an archival data storage service 110, or indicativeof the data object 114 being currently stored in an optimal location. Asa fictitious illustrative example, if a minimum requirement establishedby the object-based data storage service 104 for classifying a dataobject 114 as necessitating a transition to the archival data storageservice 110 is that the data object 114 has not been utilized by a userover a particular period of time greater than a threshold period of timeand that the frequency of use would result in a greater expense beingincurred if maintained in the current data storage tier, the machinelearning may result in a decision tree that, at least in part,bifurcates based on vector components indicating whether the period ofinactivity for the data object 114 is greater than a threshold amount oftime and whether it is cost ineffective to maintain the data object 114in the current data storage tier based at least in part on the frequencyof use of the data object 114. If the input indicates that the dataobject 114 has been inactive for a period of time greater than theperiod of time threshold and that it is cost ineffective to the customer102 to maintain the data object 114 in its current data storage tier,the one or more functions (decision trees) would, in this example,provide a result that the data object 114 is to be transitioned to thearchival data storage service 110. Thus, the machine learning algorithmmay adjust the one or more functions if these one or more functions donot indicate that the data object 114 should be transitioned to thearchival data storage service 110.

If, based at least in part on output from the machine learning algorithm(e.g., the prediction metadata), the object-based data storage service104 determines that the data object 114 is to be transitioned from thestandard data storage tier 106 to the infrequent access data storagetier 108, the object-based data storage service 104 may identify alogical data container corresponding to the infrequent access datastorage tier 108 that has sufficient capacity to support the data object114. If a logical data container is identified, the object-based datastorage service 104 may transfer the data object 114 to the logical datacontainer. Further, the object-based data storage service 104 may updatethe metadata of the logical data container to indicate that the dataobject 114 is now stored within the logical data container. In someembodiments, the object-based data storage service 104 transmits anotification to the customer 102 to indicate that the data object 114has been transferred to a logical data container corresponding to theinfrequent access data storage tier 108.

In an embodiment, if the object-based data storage service 104determines, based at least in part on usage data for the data object 114and a predicted cost assessment for storage and use of the data object114, that the data object 114 is to be stored in the archival datastorage service 110, the object-based data storage service 104 transmitsa request to the archival data storage service 110 to store the dataobject 114. In response to the request, the archival data storageservice 110 may store the data object 114 in an archive classified,through metadata, as being in an archival data storage tier 112. In anembodiment, usage data for the data object 114 is provided by thearchival data storage service 110 to the object-based data storageservice 104, which may utilize this information in conjunction with theaforementioned machine learning techniques to determine the optimalstorage location for the data object 114.

In an embodiment and as described above, the output from the machinelearning algorithm includes prediction metadata for the particular dataobject 114. The prediction metadata may specify an optimal data storagetier for the data object 114 based at least in part on the usage data.Additionally, or alternatively, the prediction metadata may provideinformation regarding storage of the data object 114 in accordance withthe various data storage tiers based at least in part on otherparameters (e.g., cost of maintaining the data object 114 in each datastorage tier in accordance with a frequency of use, etc.). Theprediction metadata may be stored along with the data object 114 in acorresponding logical data container. Alternatively, the object-baseddata storage service 104 may store the prediction metadata of the dataobject 114 in a centralized repository of the object-based data storageservice 104, in a database of the customer's account profile, or inanother location. In an embodiment, programmatic access to theprediction metadata is provided to other computer systems authorized toaccess the data object 114. This may enable these other computer systemsto utilize the prediction metadata to make programmatic decisionsregarding transitions of the data object 114 among the various datastorage tiers.

In an embodiment, the prediction metadata is utilized by theobject-based data storage service 104 to identify an initial placementfor other data objects. For instance, if the customer 102 submits arequest to the object-based data storage service 104 to store a dataobject, the object-based data storage service 104 may evaluate theprediction metadata for some or all of the customer's other dataobjects, as well as prediction metadata for similar data objects of thecustomer and/or of other customers (e.g., similar data type, similardata size, etc.), to identify an initial data storage tier for the dataobject. In an embodiment, based at least in part on prediction metadatafor similar data objects and for other data objects of the customer 102,the object-based data storage service associates the data object with asmart data storage tier, which can be utilized by the object-based datastorage service 104 as an indicator to monitor any data objects storedtherein and to identify an optimal data storage tier to minimize thecost to the customer 102 for storage and use of the data object.

The object-based data storage service 104 may also identify an initialplacement for other data objects without utilizing prediction metadata.For instance, the object-based data storage service 104 may determine aninitial placement of a data object based at least in part on usage datafor the customer's other data objects. Additionally, or alternatively,the object-based data storage service 104 may evaluate usage data forsimilar data objects, whether these similar data objects are associatedwith the customer 102 or other customers of the object-based datastorage service 104. In an embodiment, if prediction metadata is notavailable for the data object 114 and for other existing data objects,the object-based data storage service 104 generates, using theaforementioned machine learning techniques, prediction metadata forthese other existing data objects. The object-based data storage service104 may utilize this newly created prediction metadata for the otherexisting data objects to generate (e.g., by interpolation, byextrapolation, through use of other heuristics or predictive methods,etc.) prediction metadata for the new data object that is to be stored.This interpolated prediction metadata may be used to identify theinitial placement for the new data object.

If the data object 114 is transitioned to another data storage tier, theobject-based data storage service 104 may update the prediction metadatato reflect storage of the data object 114 in accordance with this otherdata storage tier. For instance, the prediction metadata may be updatedto indicate the optimal data storage tier for the data object 114 andthat the data object 114 was transitioned to this optimal data storagetier as a result of identification of the optimal data storage tierbased at least in part on the usage data for the data object 114. Asadditional usage data is obtained and processed for the data object 114,the prediction metadata may be updated to indicate the optimal datastorage tier for the data object 114 at any given time.

In an embodiment, the prediction metadata is used, along with one ormore parameters, as input to another machine learning algorithm todetermine whether to transition the data object 114 to another datastorage tier. The one or more parameters may include a constraint,specified by the customer or other authorized user, on the data storagetiers that the data object may be associated with. For example, thecustomer or other authorized user may specify that the data object 114may only be transitioned between a standard data storage tier and aninfrequent access data storage tier. Further, the customer or otherauthorized user may indicate cost limitations, which may be used toidentify whether the optimal data storage tier specified in theprediction metadata would satisfy these cost limitations. The output ofthis machine learning algorithm may include a placement decision for thedata object 114.

The object-based data storage service 104 may continue to evaluate usagedata for the data object 114 over time to determine whether a transitionof the data object 114 would result in greater cost savings for thecustomer 102 and/or for the object-based data storage service 104. Thus,if the data object 114 is utilized at a greater frequency over a periodof time, and the data object 114 is stored in a logical data containercorresponding to the infrequent access data storage tier 108 or isstored in the archival data storage service 110, the object-based datastorage service 104 may transfer the data object 114 to a logical datacontainer corresponding to the standard data storage tier 106. Thus, asthe frequency of use changes for a data object 114, the object-baseddata storage service 104 may use this frequency of use, as well as othercharacteristics (e.g., cost efficiency, availability of the logical datacontainer or data object 114 if transitioned, performance of the logicaldata container if transitioned, etc.) to determine the optimal placementof the data object 114.

FIG. 2 shows an illustrative example of a system 200 in which an accessevaluation engine 206 of an object-based data storage service 204transitions data objects among data storage tiers and an archival datastorage service 220 in accordance with at least one embodiment. In thesystem 200, an access evaluation engine 206 of the object-based datastorage service 204 monitors access and usage of various data objectsstored in logical data containers designated as being in a standard datastorage tier 208 or the infrequent access data storage tier 210. Asnoted above, a customer 202 may request storage of a data object withina logical data container corresponding to either the standard datastorage tier 208 or the infrequent access data storage tier 210. In anembodiment, the customer 202 specifies, in the request, that theobject-based data storage service 204 is to monitor usage of the dataobject to determine the optimal storage location for the data objectover time and, if an optimal storage location is identified, transferthe data object to that optimal storage location.

The access evaluation engine 206 may be implemented on a computer systemof the object-based data storage service 204, or abstraction thereof(such as one or more virtual machines operating via a hypervisor),implemented using hardware and software, and may comprise one or moreprocessors and memory that stores executable instructions whoseexecution by the one or more processors causes the computer system toperform operations described herein. For instance, the access evaluationengine 206 may monitor and record requests from the customer 202 orother users to access and utilize data objects in the logical datacontainers of the various data storage tiers. For example, if a customer202 accesses one or more data objects 216 from logical data containers212 corresponding to the standard data storage tier 208, the accessevaluation engine 206 may monitor, for each data object, the use of thedata object, the length of such usage, the time at which the data objectwas used, and the like. Similarly, if a customer 202 access one or moredata objects 218 from logical data containers 214 corresponding to theinfrequent access data storage tier 210, the access evaluation engine206 may monitor, for each data object, the use of the data object, thelength of such usage, the time at which the data object was used, andthe like. This data may be recorded in a data log that is maintained bythe access evaluation engine 206.

In an embodiment, the access evaluation engine 206 utilizes machinelearning techniques, such as supervised learning techniques to determineoptimal placement of a data object over time. For instance, the accessevaluation engine 206 may evaluate a data log corresponding to a dataobject 222 from a logical data container 212 corresponding to thestandard data storage tier 208 to identify a frequency at which the dataobject 222 has been accessed and used by the customer 202 and otherusers. Further, the access evaluation engine 206 may utilize the machinelearning techniques to evaluate the cost efficiency of maintaining thedata object 222 within the standard data storage tier 208 versustransferring the data object 222 to a logical data container 214corresponding to the infrequent access data storage tier 210 or to thearchival data storage service 220. In an embodiment, the accessevaluation engine 206 may utilize a hysteresis loop corresponding toaccess frequency and storage costs within each of the tiers and thearchival data storage service 220 to identify the optimal storagelocation for the data object. As noted above, data objects 216 in thestandard data storage tier 208 may be accessed without incurrence of afee but may be more expensive to store. However, each access of dataobjects 218 in the infrequent access data storage tier 210 may incur afee but may be less expensive to store. Similarly, storage of dataobjects in the archival data storage service 220 may be even moreinexpensive than both the infrequent access data storage tier 210 andthe standard data storage tier 208, but may require significant time toretrieve data objects, which may result in additional expense to thecustomer 202. The access evaluation engine 206 may use this informationas input to one or more machine learning algorithms to determine theoptimal placement of the data object 222.

As noted above, the output from the machine learning algorithm mayinclude prediction metadata for a particular data object being analyzedto determine optimal placement of the data object. The predictionmetadata may specify an optimal data storage tier for the data objectbased at least in part on the usage data and/or information regardingstorage of the data object in accordance with the various data storagetiers based at least in part on other parameters (e.g., cost ofmaintaining the data object in each data storage tier in accordance witha frequency of use, etc.). The access evaluation engine 206 may storethe prediction metadata along with the data object in a correspondinglogical data container or in a data archive in accordance with the datastorage tier of the data object. Alternatively, the access evaluationengine 206 may store the prediction metadata of the data object in acentralized repository of the object-based data storage service 204, ina database of the customer's account profile, or in another location. Acustomer 202 may be provided with programmatic access to the predictionmetadata via the access evaluation engine 206 or through another systemof the object-based data storage service 204. This may enable thecustomer 202 to utilize the prediction metadata to make programmaticdecisions regarding transitions of the data object among the variousdata storage tiers.

As illustrated in FIG. 2 , the access evaluation engine 206 hasdetermined that the optimal placement of the data object 222 is withinthe infrequent access data storage tier 210. Based at least in part onthis determination, the access evaluation engine 206 may transfer thedata object 222 from a logical data container 212 corresponding to thestandard data storage tier 208 to a logical data container 214corresponding to the infrequent access data storage tier 210. The accessevaluation engine 206 may update the metadata of the logical datacontainer 212 to indicate removal of the data object 222 from thelogical data container 212. Further, the access evaluation engine 206may update the metadata of the logical data container 214 to indicatestorage of the data object 222 within the logical data container 214.

Similarly, if the access evaluation engine 206 determines that theoptimal placement of a data object 224 stored in a logical datacontainer 214 corresponding to the infrequent access data storage tier210 is within the archival data storage service 220, the accessevaluation service 206 may transfer the data object 224 to the archivaldata storage service 220. The access evaluation engine 206 may transmita request to the archival data storage service 220 to store the dataobject 224 within an archive of the archival data storage service 220.The access evaluation engine 206 may also transmit requests, over time,to the archival data storage service 220 to obtain usage datacorresponding to the data object 224 and any other data objects storedwithin the archive. Based at least in part on this usage data, theaccess evaluation engine 206 may determine whether to transfer a dataobject, such as data object 228, from the archival data storage service220 to a logical data container 214 corresponding to the infrequent datastorage tier 210 or to a logical data container 212 corresponding to thestandard data storage tier 208.

If a data object, such as data object 226, that is stored in a logicaldata container 214 corresponding to the infrequent access data storagetier 210 is utilized at a frequency that would result in a greaterexpense to the customer 202 that maintaining the data object in thestandard data storage tier 208, the access evaluation engine 206 maytransfer the data object 226 from a logical data container 214corresponding to the infrequent access data storage tier 210 to alogical data container 212 corresponding to the standard data storagetier 208. The access evaluation engine 206 may update the metadata ofthe logical data container 214 to indicate removal of the data object226 from the logical data container 214. Further, the access evaluationengine 206 may update the metadata of the logical data container 212 toindicate storage of the data object 226 within the logical datacontainer 212.

To enable monitoring and automatic transitioning of data objects totheir optimal storage location, a customer of the object-based datastorage service may specify, via an interface, that the object-baseddata storage service is to monitor these data objects to determine anoptimal placement for these data objects. The object-based data storageservice may provide this interface to the customer, which may enable thecustomer to generate a logical data container and associate thisparticular logical data container with a particular data storage tier.If the customer indicates that data objects in this logical datacontainer are to be monitored for optimal placement of these dataobjects, the object-based data storage service may utilize an accessevaluation engine to monitor and record usage information for these dataobjects. Accordingly, FIG. 3 shows an illustrative example of a system300 in which an interface 306 is provided to a customer 302 to requestautomatic transitioning of data objects within a logical data containerin accordance with at least one embodiment.

In the system 300, the object-based data storage service 304 provides,to a customer 302, an interface 306 usable to request creation of alogical data container for storage of data objects and to requestmonitoring for data objects that may be stored in the requested logicaldata container. In an embodiment, the interface 306 is a GUI, althoughrequests to the object-based data storage service 304 may be submittedusing API calls or other techniques. The interface 306 may include acontainer name input field 308, which may be utilized by the customer302 to provide a name for the logical data container that is beingrequested by the customer 302. Further, the interface 306 may include adata region input field 310, which the customer 302 may use to definethe data region in which the logical data container is to be created. Insome instances, the data region input field 310 may include a drop downmenu, which may provide an ordering of the available data regionsmaintained by a computing resource service provider where the logicaldata container may be created.

The interface 306 may also include a storage tier drop down menu 322,which the customer 302 may utilize to define the corresponding datastorage tier or other data storage characteristics for the logical datacontainer. The customer 302 may select a standard data storage tier forthe logical data container if the customer 302 plans to utilize dataobjects in the logical data container frequently and requires rapidaccess to these data objects. Alternatively, if the customer 302determines that the logical data container is not going to be usedfrequently but still requires rapid access to data objects storedtherein, the customer 302 may select an option for the infrequent accessdata storage tier from the storage tier drop down menu 322. The datastorage characteristics specified via the storage tier drop down menu322 may each correspond to a set of data storage parameters. A set ofdata storage parameters may specify one or more risks corresponding tostorage of a data object amongst the various data storage tiers madeavailable by the object-based data storage service. For instance, if thecustomer selects, from the storage tier drop down menu 322, the standarddata storage tier, the object-based data storage service may determinethat the customer assumes the risk of incurring greater costs if a dataobject stored in a logical data container corresponding to the standarddata storage tier is utilized infrequently or at a rate that would makeit more expensive to the customer versus storing the data object inanother data storage tier. As another example, if the customer selectsthe infrequent access data storage tier from the storage tier drop downmenu 322, the object-based data storage tier may determine that thecustomer assumes the risk of incurring greater costs if a data objectstored in a logical data container corresponding to the infrequentaccess data storage tier is accessed at a rate that would result ingreater cost to the customer versus storing the data object in thestandard data storage tier. In an embodiment, the object-based datastorage service utilizes the set of data storage parameters to determinethe data storage tier for storage of the data object. Other risks thatmay be specified in the set of data storage parameters include risksassociated with potential loss of data or of the data object itself,availability of resources corresponding to the data storage tiers forstorage of the data object, potential delays in availability of theseresources, likelihood of failures of the software and/or hardware layersof the object-based data storage service, and the like.

In an embodiment, the interface 306 includes one or more radio buttons312 that can be used by the customer 302 to enable automatictransitioning of data objects stored in the logical data container thatis to be created. If the customer 302 uses the one or more radio buttons312 to indicate that the object-based data storage service 304 mayautomatically transition data objects from the logical data container toanother data storage tier, the object-based data storage service utilizethe characteristics of these data objects and cost efficiencyinformation to identify the optimal placement of these data objects overtime.

The interface 306 may also include a transition parameter drop down menu314, which may enable the customer 302 to define the authorizedtransitions that may be performed by the object-based data storageservice 304 for data objects stored in the logical data container. Forinstance, as illustrated in FIG. 3 , the customer 302 may select anoption via the transition parameter drop down menu 314 that would enablethe object-based data storage service 304 to transition data objectsamong the standard data storage tier, the infrequent access data storagetier, and an archival data storage service as needed based at least inpart on the characteristics of the data objects and cost efficiencyinformation for the data objects. Through the transition parameter dropdown menu 314, the customer 302 may limit the available transitions bythe object-based data storage service 304. For example, if the logicaldata container and corresponding data objects are maintained in thestandard data storage tier, the customer 302 may specify that dataobjects may only be transitions between the standard data storage tierand the infrequent access data storage tier. The elements of thetransition parameter drop down menu 314 may change based at least inpart on the storage tier selected via the storage tier drop down menu322. For example, if the customer 302 has selected the standard datastorage tier from the storage tier drop down menu 322, the optionspresented via the transition parameter drop down menu 314 may includethe standard data storage tier. In an embodiment, if the customer 302specifies that automatic transitioning for data objects is not to beperformed, the interface 306 may omit the transition parameter drop downmenu 314 and not present this to the customer 302. In an embodiment, ifthe customer 302 selects an option via the transition parameter dropdown menu 314 to enable the object-based data storage service 304 totransition data objects among the standard data storage tier, theinfrequent access data storage tier, and an archival data storageservice, the object-based data storage service 304 incorporates thisoption into the data storage parameters described above. This may enablethe object-based data storage service 304 to utilize the set of datastorage parameters for the data object to determine the initial datastorage tier for the data object and as input, along with usage data forthe data object, to the machine learning algorithms to determine whetherto transition the data object to another data storage tier.

The interface 306 may further include one or more radio buttons 316 thatcan be used by the customer 302 to authorize the object-based datastorage service 304 to utilize usage data for data objects in thelogical data container for various purposes. For instance, this usagedata may be collected and utilized by the object-based data storageservice 304, along with the data object characteristics and costefficiency information, to determine whether these data objects areplaced in an optimal location and, if not, to transfer these dataobjects to the optimal location. Further, this usage data may be used inaggregate with usage data for other data objects as a reference formaking determinations regarding transitions for other sets of dataobjects that may be monitored by the object-based data storage service304.

To submit the request to generate the logical data container and toenable monitoring and transitioning of the data objects stored therein,the customer 302 may select a submit button 318 provided via theinterface. The object-based data storage service 304 may receive therequest and generate the logical data container in the specified datastorage tier. Further, if the customer 302 has specified, via theinterface 306, that data objects of the logical data container are to bemonitored and transitioned to other data storage tiers based at least inpart on usage data for these data objects, the object-based data storageservice 304 may perform these operations for these data objects. If thecustomer 302 selects the cancel button 320 from the interface 306, theinterface 306 may terminate without providing a request to theobject-based data storage service 304.

In an embodiment, the object-based data storage service provides a smartdata storage tier to customers of the object-based data storage serviceto enable the object-based data storage service to automaticallydetermine the optimal data storage location for each data object addedto the smart data storage tier. Thus, a customer may not be required tospecify an initial data storage tier for a data object, as theobject-based data storage service may determine, based at least in parton the characteristics of the data object, the customer's use of otherdata objects, and usage data for similar data objects, an initial datastorage location for the data object. Accordingly, FIG. 4 shows anillustrative example of a system 400 in which an object placement engine410 of an object-based data storage service 404 determines placement ofa data object from a smart data storage tier 406 in accordance with atleast one embodiment.

In the system 400, a customer 402 of the object-based data storageservice 404 submits a request to the object-based data storage service404 to store a data object 408 in a logical data container correspondingto a smart data storage tier 406. The smart data storage tier 406 may beutilized by the object-based data storage service 404 as an indicator tomonitor any data objects stored therein and to identify an optimal datastorage tier to minimize the cost to the customer 402 for storage anduse of the data object 408. Thus, while the standard data storage tier414, the infrequent access data storage tier 416, and the archival datastorage tier 418 may be associated with a particular storage cost andusage cost for data objects, the smart data storage tier 406 may nothave a storage cost or usage cost associated with placement of the dataobject 408 in the smart data storage tier 406.

If the customer 402 stores a data object 408 in a logical data containercorresponding to the smart data storage tier 406, an initial placementrequest is sent by a computer system of the smart data storage tier 406to an object placement engine 410 to identify a data storage tier wherethe data object 408 may be stored and monitored. The object placementengine 410 may be implemented on a computer system of the object-baseddata storage service 404, or abstraction thereof (such as one or morevirtual machines operating via a hypervisor), implemented using hardwareand software, and may comprise one or more processors and memory thatstores executable instructions whose execution by the one or moreprocessors causes the computer system to perform operations describedherein. In response to the placement request, the object placementengine 410 may access, from a customer profile datastore 412, thecustomer's profile to identify any other data objects stored on behalfof the customer 402 in any other data storage tiers (e.g., standard datastorage tier 414, infrequent access data storage tier 416, archival datastorage tier 418, etc.). The object placement engine 410 may evaluateeach of these data objects to identify the characteristics of these dataobjects. Further, the object placement engine 410 may obtain usage datafor each of these data objects from the access evaluation enginedescribed above.

Utilizing the usage data from the access evaluation engine and thecharacteristics of existing data objects of the customer 402 as input toone or more machine learning algorithms, the object placement engine 410may identify an initial placement location for the data object 408. Themachine learning algorithms utilized by the object placement engine 410may be similar to those described above in connection with the accessevaluation engine except that these machine learning algorithms mayadditionally utilize the characteristics of existing data objects of thecustomer 402 as input. The output of the machine learning algorithms maybe a placement decision corresponding to the initial placement locationof the data object 408. The object placement engine 410, based at leastin part on the placement decision, may transfer the data object 408 to alogical container of the standard data storage tier 414 or of theinfrequent access data storage tier 416 or to an archive of the archivaldata storage tier 418.

In an embodiment, the data object 408 is stored with metadata specifyingthat the data object 408 is a member of the smart data storage tier 406.Thus, if the customer 402 opts to evaluate the placement of the dataobject 408 via an interface of the object-based data storage service404, the data object 408 may be presented as being in the smart datastorage tier 406 and not in any other data storage tier, even though thedata object 408, for purposes of fee calculation, may be stored in alogical data container corresponding to another data storage tier.

The object placement engine 410 may further evaluate, for an existingdata object of the smart data storage tier 406, any new usage data forthe data object and characteristics of existing data objects 420, 422,424 in the various data storage tiers 414, 416, 418 to determine whetherthe optimal storage location for the data object differs from itscurrent storage location. If the object placement engine 410 determinesa new optimal storage location for the data object, the object placementengine 410 may transfer the data object from its current data storagetier to a logical data container corresponding to the optimal datastorage tier. In an embodiment, the metadata for the data objectmaintains an association with the smart data storage tier 406 regardlessof how the data object is transitioned by the object placement engine410. The evaluation of an existing data object may be similar to thatperformed by the access evaluation engine 206 discussed above inconnection with FIG. 2 , except that the object placement engine 410 mayalso evaluate usage data for other data objects classified as being inthe smart data storage tier 406, as well as the characteristics of thesedata objects, to generate a placement decision for an existing dataobject classified as being in the smart data storage tier 406.

In an embodiment, the object-based data storage service provides, via aninterface, an option to customers of the object-based data storageservice to generate a logical data container that can be classified asbeing in the smart data storage tier. Data objects stored within thislogical data container may also be classified as being in the smart datastorage tier and may cause the object-based data storage service toidentify an optimal data storage tier for these data objects for thepurpose of optimizing savings for the customer in the storage of thesedata objects. Accordingly, FIG. 5 shows an illustrative example of asystem 500 in which an interface 506 is provided to a customer 502 torequest creation of a logical data container in a smart data storagetier to enable optimized placement of data objects of the logical datacontainer in accordance with at least one embodiment.

In the system 500, the object-based data storage service 504 provides,to a customer 502, an interface 506 usable to request creation of alogical data container for storage of data objects. In an embodiment,the interface 506 is a GUI, although requests to the object-based datastorage service 504 may be submitted using API calls or othertechniques. The interface 506 may include similar fields as those of theinterface 306 described above in connection with FIG. 3 . For example,the interface 506 may include a container name input field 508, whichmay be utilized by the customer 502 to provide a name for the logicaldata container that is being requested by the customer 502.

In an embodiment, the interface 506 includes a storage tier drop downmenu 510, which the customer 502 may utilize to define the correspondingdata storage tier or other data storage characteristics for the logicaldata container. Through use of the storage tier drop down menu 510, thecustomer 502 may select a smart data storage tier for the logical datacontainer if the customer 502 wants to enable the object-based datastorage service 504 to make all placement decisions with regard to dataobjects added to the logical data container in the smart data storagetier. Alternatively, if the customer 502 wants to define the initialstorage tier for the logical data container and corresponding dataobjects, the customer 502 may select the standard data storage tier orthe infrequent access data storage tier from the storage tier drop downmenu 510.

In an embodiment, if the customer 502 selects the smart data storagetier from the storage tier drop down menu 510, the object-based datastorage service 504 may omit presentation of other elements of theinterface 506 that may be presented upon selection of a different datastorage tier from the storage tier drop down menu 510. For example, ifthe customer 502 selects the smart data storage tier option from thestorage tier drop down menu 510, the object-based data storage servicemay omit the one or more radio buttons 312, as illustrated in FIG. 3 ,that can be used by the customer 502 to enable automatic transitioningof data objects stored in the logical data container that is to becreated. Further, the object-based data storage service 504 may omit atransition parameter drop down menu, which may typically be presented toenable the customer 502 to define the authorized transitions that may beperformed by the object-based data storage service 504 for data objectsstored in the logical data container. Similarly, the object-based datastorage service 504 may omit the one or more radio buttons that can beused by the customer 502 to authorize the object-based data storageservice 504 to utilize usage data for data objects in the logical datacontainer for various purposes. Thus, by selecting the smart datastorage tier option from the storage tier drop down menu 510, thecustomer 502 may grant the object-based data storage service 504 tomonitor data objects in the smart data storage tier and to automaticallytransition these data objects to optimal placement locations based atleast in part on the usage data for the data objects and for othersimilar data objects, identified based at least in part on thecharacteristics of the data objects in the smart data storage tier.

Similar to the storage tier drop down menu 322 described above inconnection with FIG. 3 , the data storage characteristics specified viathe storage tier drop down menu 510 may each correspond to a set of datastorage parameters. These sets of data storage parameters may eachspecify one or more risks corresponding to storage of a data objectamongst the various data storage tiers made available by theobject-based data storage service. In an embodiment, if the customerselects, from the storage tier drop down menu 510, the smart datastorage tier, the object-based data storage service determines that theobject-based data storage service assumes the risk of incurring greatercosts if a data object stored in a logical data container correspondingto the standard data storage tier is utilized infrequently or at a ratethat would make it more expensive to the object-based data storageservice versus storing the data object in another data storage tier. Asanother example, if the customer selects the smart data storage tierfrom the storage tier drop down menu 510, the object-based data storagetier may assume the risk of incurring greater costs if a data objectstored in a logical data container corresponding to the infrequentaccess data storage tier is accessed at a rate that would result ingreater cost to the object-based data storage service versus storing thedata object in the standard data storage tier. Thus, if the customer 502selects, from the storage tier drop down menu 510, the smart datastorage tier, the customer 502 may incur a fixed cost for storage of thedata object while the object-based data storage service assumes therisks associated with access to the data object in the various datastorage tiers (e.g., standard, infrequent access, archival, etc.). In anembodiment, the object-based data storage service utilizes the set ofdata storage parameters to determine the initial data storage tier forstorage of the data object.

In an embodiment, selection of the smart data storage tier via thestorage tier drop down menu 510 serves as an authorization, granted bythe customer 502 to the object-based data storage service, to enableselection of an initial data storage tier (e.g., standard, infrequentaccess, archival, etc.) from the plurality of data storage tiers for thedata object. Further, selection of the smart data storage tier may alsoserve as an authorization to transition the data object among theplurality of data storage tiers as determined by the object-based datastorage service based at least in part on usage data for the data objectand the set of data storage parameters (e.g., risks of maintaining adata object in a more expensive data storage tier based at least in parton the usage data, etc.).

If the customer 502 selects another option from the storage tier dropdown menu 510, the object-based data storage service 504 may revise theinterface 506 to include the features described above in connection withthe interface 306 illustrated in FIG. 3 . If the customer 502 selectsthe smart data storage tier option from the storage tier drop down menu510, the customer 502 may select the submit button 512 to submit arequest to the object-based data storage service 504 to generate thelogical data container in the smart data storage tier. However, if thecustomer 502 wishes to cancel its request, the customer 502 may selectthe cancel button 514, which may cause the object-based data storageservice 504 to terminate the interface 506 without obtaining the requestfrom the customer 502.

FIG. 6 shows an illustrative example of a system 600 in which historicalaccess data for data objects 612 stored in a smart data storage tier 608is utilized to determine a placement for the data objects 612 inaccordance with at least one embodiment. In the system 600, a customer602 of the object-based data storage service 604 submits a request tostore one or more data objects 612 in a logical data container 610corresponding to a smart data storage tier 608. In response to therequest, the object placement engine 618 accesses the customer's profilefrom a customer profile datastore 620 to identify the customer's otherdata objects that may be stored in a logical data container 610corresponding to the smart data storage tier 608, in other data storagetiers (e.g., standard data storage tier 614, infrequent access datastorage tier 616), and in the archival data storage service 622. Theobject placement engine 618 may obtain, from the access evaluationengine 606, usage data corresponding to these data objects.

In an embodiment, the object placement engine 618 evaluates the one ormore data objects 612 that the customer has stored in the smart datastorage tier 608 to identify the characteristics of these data objects612. For instance, the object placement engine 618 may identify the sizeof each data object, the age of each data object, the type of dataobject, and the like. This information may be used by the objectplacement engine 618 to identify similar data objects stored in thesmart data storage tier 608, in the other data storage tiers, and in thearchival data storage service 622. Further, the object placement engine618 may obtain, from the access evaluation engine 606, usage data forthese similar data objects.

The object placement engine 618 may utilize the usage data for thesimilar data objects and the customer's other data objects and thecharacteristics of the data object added to the smart data storage tier608 as input to one or more machine learning algorithms to identify anoptimal storage location for the data object. For instance, if theobject placement engine 618 determines that a data object 624 added tothe smart data storage tier 608 by the customer 602 is similar to otherdata objects stored in logical data containers corresponding to thestandard data storage tier 614 and that the customer 602 utilizes itsother data objects at a high frequency, the object placement engine 618may store the data object 624 in association with the standard datastorage tier 614. For example, the object placement engine 618 maytransition the data object 624 to a logical data container correspondingto the standard data storage tier 614. Further, the object placementengine 618 may update metadata of the data object 624 to indicate thatthe data object 624 is associated with the smart data storage tier 608.This information may also be included in the customer's profile. Thus,if the customer 602 submits a request to access the data object 624, theobject-based data storage service 604 may indicate that this data object624 is in the smart data storage tier 608 instead of being in thestandard data storage tier 614.

As another example, if the object placement engine 618 determines that adata object 626 added to the smart data storage tier 608 by the customer602 is similar to other data objects stored in logical data containerscorresponding to the infrequent access data storage tier 616 and thatthe customer 602 utilizes its other data objects at a lesser rate, theobject placement engine 618 may store the data object 626 in associationwith the standard data storage tier 616. The object placement engine 618may transition the data object 626 to a logical data containercorresponding to the infrequent access data storage tier 616 and mayupdate metadata of the data object 626 to indicate that the data object626 is associated with the smart data storage tier 608. Similarly, ifthe object placement engine 618 determines that a data object 628 addedto the smart data storage tier 608 is similar to other data objectsstored in archives of the archival data storage service 622 and that thecustomer 602 is unlikely to utilize the data object 628 at a significantrate (e.g., the data object 628 includes backup data, etc.), the objectplacement engine 618 may transition the data object 628 to an archive ofthe archival data storage service 622.

In an embodiment, once an initial placement decision is made for a dataobject from the smart data storage tier 608, the access evaluationengine 606 monitors and records requests from the customer 602 or otherusers to access and utilize data objects in the logical data containersof the various data storage tiers. For example, if a customer 602accesses one or more data objects from logical data containerscorresponding to the standard data storage tier 614, the accessevaluation engine 606 may monitor, for each data object, the use of thedata object, the length of such usage, the time at which the data objectwas used, and the like. Similarly, if a customer 602 access one or moredata objects from logical data containers corresponding to theinfrequent access data storage tier 616, the access evaluation engine606 may monitor, for each data object, the use of the data object, thelength of such usage, the time at which the data object was used, andthe like. This data may be recorded in a data log that is maintained bythe access evaluation engine 606.

The object placement engine 618 may obtain the data recorded in the datalog to evaluate the usage of the data objects 612 of the smart datastorage tier 608, along with the characteristics of these data objects612 and those of similar data objects, to determine whether these dataobjects 612 are stored in their optimal placement locations. Forexample, if the object placement engine 618 determines that a dataobject, stored in the standard data storage tier 614, is being utilizedat a lower frequency than that of other data objects in the standarddata storage tier 614, the object placement engine 618 may transfer thedata object to the infrequent access data storage tier 616. Similarly,if the frequency at which the data object 630 is utilized is below aparticular usage threshold, the object placement engine 618 may transferthe data object 630 to the archival data storage service 622. The objectplacement engine 618 may maintain the metadata for the data object 630to maintain an association between the data object 630 and the smartdata storage tier 608. Thus, even if the data object is transitionedbetween tiers and the archival data storage service 622 (e.g., dataobject 630, data object 632, etc.), the customer 602 is presented withthe data object as if the data object is in a logical data container 610of the smart data storage tier 608.

FIG. 7 shows an illustrative example of a process 700 for monitoringusage of a data object to determine optimal placement of the data objectin response to a request from a customer to automatically transition thedata object in accordance with at least one embodiment. The process 700may be performed by a computer system or abstraction thereof (such asone or more virtual machines operating via a hypervisor) of theobject-based data storage service, such as an access evaluation engine,described herein. In an embodiment, the access evaluation engine detects702 storage of a data object in a logical data container of a particulardata storage tier. This data storage tier may correspond to either thestandard data storage tier or the infrequent access data storage tiersdescribed above. A request to store the data object in a particular datastorage tier may be submitted by a customer of the object-based datastorage service via an interface, such as the interface 306 describedabove in connection with FIG. 3 . Through this interface, the customermay specify whether the object-based data storage service may monitorusage of this data object and whether the object-based data storageservice may automatically transition data objects between these datastorage tiers and/or an archival data storage service. Thus, based atleast in part on the customer inputs via the interface, the accessevaluation engine may determine 704 whether monitoring of the dataobject has been specified.

If the customer has specified that the object-based data storage serviceis not to monitor usage of the data object, the access evaluationservice may maintain 706 the data object in the data storage tier inwhich the data object is stored. Thus, regardless of any variation inusage, the data object may remain in its original data storage tier.However, if the customer has indicated that the object-based datastorage service is to monitor usage of the data object, the accessevaluation engine may monitor 708 and record any access to the dataobject in a data log corresponding to the data object.

The access evaluation engine may analyze 710 these data logs to identifyhistorical access to the data object over time. Using these data logs,the access evaluation engine may determine the frequency at which thedata object is accessed by the customer and other users. Further, insome embodiments, the access evaluation engine evaluates other dataobjects in the data storage tier to compare the usage frequency of thesedata objects to that of the data object stored by the customer in thedata storage tier. In an embodiment, the access evaluation engineutilizes this information as input into one or more machine learningalgorithms, as described herein, to obtain an output corresponding tothe optimal storage location for the data object. Thus, based at leastin part on this output and the data obtained through evaluation of thedata logs, the access evaluation engine may determine 712 whether accessto the data object over time is consistent with the desired usagefrequency for storage in the current data storage tier.

If the access evaluation engine determines that access to the dataobject over time is consistent with the desired usage frequency forstorage in the current data storage tier, the access evaluation enginemay maintain the data object in its current data storage tier andcontinue to monitor 708 further accesses to the data object. However, ifthe access evaluation engine determines that usage of the data object isinconsistent with the expected usage patterns for data objects withinthe data storage tier, the access evaluation engine may transfer 714 thedata object to a different data storage tier. For instance, if the dataobject is stored in a standard data storage tier but the accessevaluation engine determines that the data object is being utilized at afrequency that is below a threshold for the standard data storage tier,the access evaluation engine may transfer the data object to aninfrequent access data storage tier or to an archival data storageservice. The access evaluation engine may further monitor 708 accessesto the data object in its new data storage tier. This enables continuousoptimization of storage for the data object to minimize the storagecosts to the customer.

It should be noted that the process 700 may be performed usingadditional and/or alternative operations to those illustrated in FIG. 7. For example, in an embodiment, the output from the analysis of thedata logs and the determination regarding usage of the data object caninclude prediction metadata for the data object. The prediction metadatamay specify the optimal data storage tier for the data object based atleast in part on the usage data and/or information regarding storage ofthe data object in accordance with the various data storage tiers basedat least in part on other parameters (e.g., cost of maintaining the dataobject in each data storage tier in accordance with a frequency of use,etc.). The access evaluation engine may store the prediction metadataalong with the data object in a corresponding logical data container orin a data archive in accordance with the data storage tier of the dataobject. Alternatively, the access evaluation engine may store theprediction metadata of the data object in a centralized repository ofthe object-based data storage service, in a database of the customer'saccount profile, or in another location. The access evaluation enginemay provide programmatic access to the prediction metadata to othercomputer systems, such as a customer computer system. This may enable acustomer to utilize the prediction metadata to make programmaticdecisions regarding transitions of the data object among the variousdata storage tiers.

As noted above, the object-based data storage service may provide asmart data storage tier for storage of various data objects. Placementof a data object in the smart data storage tier may serve as anindication to the object-based data storage service that theobject-based data storage service is to monitor usage of the data objectand utilize the characteristics of the data object (e.g., size, age,type, etc.) to identify an optimal placement for the data object withinone of the fee-based tiers (e.g., standard data storage tier, infrequentaccess data storage tier) or the archival data storage service. Thus, ifa customer selects the smart data storage tier for storage of a dataobject, the customer need not provide an explicit indication of afee-based data storage tier for storage of the data object, nor does thecustomer need to indicate whether monitoring of the data object ispermitted. Accordingly, FIG. 8 shows an illustrative example of aprocess 800 for utilizing usage data for other data objects in a smartdata storage tier to determine initial placement of a data object inresponse to a request to store the data object in the smart data storagetier in accordance with at least one embodiment. The process 800 may beperformed by a computer system or abstraction thereof (such as one ormore virtual machines operating via a hypervisor) of the object-baseddata storage service, such as an object placement engine, describedherein.

At any time, the object-based data storage service may receive 802 arequest to store a data object in a logical data container correspondingto a particular data storage tier. In an embodiment, the object-baseddata storage service provides an interface to customers, such as theinterfaces described above in connection with FIGS. 3 and 5 . Throughthe interface, a customer may select which data storage tier the dataobject is to be stored in, as well as provide an indication as towhether the object-based data storage service is to automaticallytransition the data object to other data storage tiers or to an archivaldata storage service based at least in part on an analysis of the usageof the data object over time. In an embodiment, if the customer selectsa fee-based data storage tier, such as the standard data storage tier orthe infrequent access data storage tier, the object-based data storageservice provides the customer with various options via the interface.For example, as illustrated and described above in connection with FIG.3 , the object-based data storage service may allow customer to enablemonitoring and automatic transitioning of the data object to an optimallocation based at least in part on its usage. Further, the object-baseddata storage service may allow the customer to restrict thesetransitions to certain data storage tiers. For instance, the customermay define that transitions may only be made between the standard datastorage tier and the infrequent access data storage tier. This wouldprevent the object-based data storage service from transitioning thedata object to the archival data storage service.

In an embodiment, the object-based data storage service, via theinterface, provides an option to the customer to store the data objectin a logical data container corresponding to a smart data storage tier.Unlike the fee-based data storage tiers described above, the smart datastorage tier may serve as an implicit indication that the object-baseddata storage service is to automatically determine initial andsubsequent placement of the data object within the various fee-baseddata storage tiers while maintaining an association between the dataobject and the smart data storage tier. Thus, if the customer selectsthe smart data storage tier option from the interface, the customer maynot be required to indicate which transitions are permitted or toindicate that monitoring of the data object is authorized as selectionof the smart data storage tier may serve as an implicit indication ofthe customer's authorization of the object-based data storage service tomonitor and automatically transition the data object based at least inpart on the characteristics of the data object and the usage of the dataobject over time. Hence, the object-based data storage service maydetermine 804 whether the customer has specified that the data object bestored in a logical data container corresponding to the smart datastorage tier.

If the customer has selected a different data storage tier, theobject-based data storage service may store 806 the data object in alogical data container corresponding to the selected data storage tier.If the customer has specified that monitoring of the data object isauthorized, the object-based data storage service may cause the accessevaluation engine to initiate monitoring of the data object.Alternatively, if the customer has indicated, via the interface, thatmonitoring and automatic transitioning of the data object is notauthorized, the object-based data storage service may maintain the dataobject in the selected data storage tier.

If the customer has selected the smart data storage tier, theobject-based data storage service may cause an object placement engineto evaluate 808 usage data for other data objects in the smart datastorage tier. For example, the object-based data storage service mayevaluate the data object that is to be stored in the smart data storagetier to identify various characteristics of the data object. This mayinclude the size of the data object, the type of the data object (e.g.,file type, etc.), the age of the data object if the data object (e.g.,date the data object was created, etc.), and the like. Using thesecharacteristics, the object placement engine may identify similar dataobjects that may be associated with the smart data storage tier.Further, the object placement engine may identify any of the customer'sother data objects maintained by the object-based data storage serviceor the archival data storage service. The object placement engine mayobtain usage data for each of these other data objects and identify anyaccess patterns for these data objects. This information may be used tosupplement the set of data storage parameters of the data object. Thisset of data storage parameters may be used as input to one or moremachine learning algorithms to obtain an initial placement decision forthe newly added data object. Thus, based at least in part on thisplacement decision, the object placement engine identifies 810 afee-based data storage tier for the data object.

The object placement engine, as a result of having identified afee-based data storage tier for the data object, may store 812 the dataobject in a storage location corresponding to the identified fee-baseddata storage tier. For instance, if the object placement enginedetermines that the data object is to be stored in a locationcorresponding to the standard data storage tier, the object placementengine may identify a logical data container corresponding to thestandard data storage tier that has capacity to enable storage of thedata object. The object placement engine may update metadata of thelogical data container to indicate storage of the data object therein.Further, the object placement engine maintains metadata for the dataobject that specifies that the data object is associated with the smartdata storage tier. This may cause an access evaluation engine to monitoraccess and usage of the data object. The monitoring of the data objectmay result in data logs that may be used to determine whether the dataobject is to be transitioned to another fee-based data storage tier orto the archival data storage service, as needed.

As noted above, a data object associated with the smart data storagetier may be stored in a logical data container corresponding to afee-based data storage tier or in an archive of an archival data storageservice. The object-based data storage service, using an accessevaluation engine, may monitor access and usage of the data object overtime, which may be recorded in one or more data logs. Further, an objectplacement engine may utilize these logs, as well as other informationregarding the data object and similar data objects, may identify anoptimal storage location for the data object and transition the dataobject to this optimal storage location, as needed. Accordingly, FIG. 9shows an illustrative example of a process 900 for utilizingcharacteristics and access patterns for an existing data object in asmart data storage tier to determine an optimal storage location for thedata object in accordance with at least one embodiment. The process 900may be performed by a computer system or abstraction thereof (such asone or more virtual machines operating via a hypervisor) of theobject-based data storage service, such as an object placement engine,described herein.

The object placement engine may obtain 902, for a data object associatedwith the smart data storage tier, usage data from an access evaluationengine of the object-based data storage service. The object placementengine may perform this operation periodically, such as at certain timeintervals (e.g., every hour, every day, every week, etc.) or in responseto a triggering event (e.g., change in availability of computingresources associated with a particular data storage tier, etc.). Inaddition to obtaining the usage data for the data object, the objectplacement engine may obtain 904 one or more data logs that specify dataregarding historical access over time to the data object. For instance,the usage data may specify a contemporaneous detailing of the usage ofthe data object by the customer and other users. The data logs maysupplement this information by providing details regarding the frequencyat which the data object has been used over an extended period of time,as well as the duration of each use and the purpose for each use.

In addition to obtaining the usage data and data logs for the dataobject, the object placement engine may evaluate 906 the characteristicsof the data object and utilize the usage data and historical access datato identify one or more access patterns for the data object. Theseaccess patterns may correspond to periods of inactivity and periods ofelevated activity for the data object. This information may be utilizedto supplement the set of data storage parameters for the data object, asdescribed below, to identify an optimal placement for the particulardata object during these identified periods.

The object placement engine may identify 908 the current storagelocation for the data object. As described above, although the dataobject may be associated with a set of data storage characteristicsclassified as being the smart data storage tier, the data object may bestored in a logical data container corresponding to a fee-based datastorage tier (e.g., standard data storage tier, infrequent access datastorage tier, etc.) or in an archive of an archival data storageservice. Thus, the object placement engine may utilize metadata of thedata object to determine the storage location of the data object and thecorresponding data storage tier that it is located in. In an embodiment,the object placement engine utilizes the usage data and the set of datastorage parameters, which may include the associated risks formaintaining the data object in the current storage location, historicalaccess data, the characteristics of the data object, and the accesspatterns for the data object, as input to one or more machine learningalgorithms to determine an optimal data storage location for the dataobject. Using the output of these algorithms, the object placementengine may determine 910 whether the data object is in an optimalstorage location (e.g., in a storage location corresponding to theoptimal data storage tier for the data object).

If the data object is already in the optimal storage location, theobject placement engine may continue to analyze the characteristics andthe data associated with the data object to determine whether the dataobject is to be transitioned to another storage location at anothertime. However, if the object placement engine determines that the dataobject is not in its optimal storage location, the object placementengine may transition 912 the data object to this new storage location.For instance, the object placement engine may modify metadata associatedwith a logical data container corresponding to the optimal data storagetier to indicate storage of the data object within the logical datacontainer. Further, the object placement engine may revise the metadataof the former logical data container that stored the data object toindicate removal of the data object from the logical data container.Once the data object has been transitioned to the new storage location,the object placement engine may continue to analyze the characteristicsand the data associated with the data object to determine whether thedata object is to be transitioned to another storage location as needed.Thus, the object placement engine may transition the data object asneeded and ensure optimized placement on behalf of the customer, withoutthe customer needed to indicate any transition parameters for the dataobject.

FIG. 10 illustrates aspects of an example system 1000 for implementingaspects in accordance with an embodiment. As will be appreciated,although a web-based system is used for purposes of explanation,different systems may be used, as appropriate, to implement variousembodiments. In an embodiment, the system includes an electronic clientdevice 1002, which includes any appropriate device operable to sendand/or receive requests, messages, or information over an appropriatenetwork 1004 and convey information back to a user of the device.Examples of such client devices include personal computers, cellular orother mobile phones, handheld messaging devices, laptop computers,tablet computers, set-top boxes, personal data assistants, embeddedcomputer systems, electronic book readers, and the like. In anembodiment, the network includes any appropriate network, including anintranet, the Internet, a cellular network, a local area network, asatellite network or any other such network and/or combination thereofand components used for such a system depend at least in part upon thetype of network and/or system selected. Many protocols and componentsfor communicating via such a network are well known and will not bediscussed herein in detail. In an embodiment, communication over thenetwork is enabled by wired and/or wireless connections and combinationsthereof. In an embodiment, the network includes the Internet and/orother publicly-addressable communications network, as the systemincludes a web server 1006 for receiving requests and serving content inresponse thereto, although for other networks an alternative deviceserving a similar purpose could be used as would be apparent to one ofordinary skill in the art.

In an embodiment, the illustrative system includes at least oneapplication server 1008 and a data store 1010 and it should beunderstood that there can be several application servers, layers orother elements, processes or components, which may be chained orotherwise configured, which can interact to perform tasks such asobtaining data from an appropriate data store. Servers, in anembodiment, are implemented as hardware devices, virtual computersystems, programming modules being executed on a computer system, and/orother devices configured with hardware and/or software to receive andrespond to communications (e.g., web service application programminginterface (API) requests) over a network. As used herein, unlessotherwise stated or clear from context, the term “data store” refers toany device or combination of devices capable of storing, accessing andretrieving data, which may include any combination and number of dataservers, databases, data storage devices and data storage media, in anystandard, distributed, virtual or clustered system. Data stores, in anembodiment, communicate with block-level and/or object level interfaces.The application server can include any appropriate hardware, softwareand firmware for integrating with the data store as needed to executeaspects of one or more applications for the client device, handling someor all of the data access and business logic for an application.

In an embodiment, the application server provides access controlservices in cooperation with the data store and generates contentincluding, but not limited to, text, graphics, audio, video and/or othercontent that is provided to a user associated with the client device bythe web server in the form of HyperText Markup Language (“HTML”),Extensible Markup Language (“XML”), JavaScript, Cascading Style Sheets(“CSS”), JavaScript Object Notation (JSON), and/or another appropriateclient-side or other structured language. Content transferred to aclient device, in an embodiment, is processed by the client device toprovide the content in one or more forms including, but not limited to,forms that are perceptible to the user audibly, visually and/or throughother senses. The handling of all requests and responses, as well as thedelivery of content between the client device 1002 and the applicationserver 1008, in an embodiment, is handled by the web server using PHP:Hypertext Preprocessor (“PHP”), Python, Ruby, Perl, Java, HTML, XML,JSON, and/or another appropriate server-side structured language in thisexample. In an embodiment, operations described herein as beingperformed by a single device are performed collectively by multipledevices that form a distributed and/or virtual system.

The data store 1010, in an embodiment, includes several separate datatables, databases, data documents, dynamic data storage schemes and/orother data storage mechanisms and media for storing data relating to aparticular aspect of the present disclosure. In an embodiment, the datastore illustrated includes mechanisms for storing production data 1012and user information 1016, which are used to serve content for theproduction side. The data store also is shown to include a mechanism forstoring log data 1014, which is used, in an embodiment, for reporting,computing resource management, analysis or other such purposes. In anembodiment, other aspects such as page image information and accessrights information (e.g., access control policies or other encodings ofpermissions) are stored in the data store in any of the above listedmechanisms as appropriate or in additional mechanisms in the data store1010.

The data store 1010, in an embodiment, is operable, through logicassociated therewith, to receive instructions from the applicationserver 1008 and obtain, update or otherwise process data in responsethereto and the application server 1008 provides static, dynamic, or acombination of static and dynamic data in response to the receivedinstructions. In an embodiment, dynamic data, such as data used in weblogs (blogs), shopping applications, news services, and other suchapplications are generated by server-side structured languages asdescribed herein or are provided by a content management system (“CMS”)operating on, or under the control of, the application server. In anembodiment, a user, through a device operated by the user, submits asearch request for a certain type of item. In this example, the datastore accesses the user information to verify the identity of the user,accesses the catalog detail information to obtain information aboutitems of that type, and returns the information to the user, such as ina results listing on a web page that the user views via a browser on theuser device 1002. Continuing with example, information for a particularitem of interest is viewed in a dedicated page or window of the browser.It should be noted, however, that embodiments of the present disclosureare not necessarily limited to the context of web pages, but are moregenerally applicable to processing requests in general, where therequests are not necessarily requests for content. Example requestsinclude requests to manage and/or interact with computing resourceshosted by the system 1000 and/or another system, such as for launching,terminating, deleting, modifying, reading, and/or otherwise accessingsuch computing resources.

In an embodiment, each server typically includes an operating systemthat provides executable program instructions for the generaladministration and operation of that server and includes acomputer-readable storage medium (e.g., a hard disk, random accessmemory, read only memory, etc.) storing instructions that, if executed(i.e., as a result of being executed) by a processor of the server,cause or otherwise allow the server to perform its intended functions.

The system 1000, in an embodiment, is a distributed and/or virtualcomputing system utilizing several computer systems and components thatare interconnected via communication links (e.g., transmission controlprotocol (TCP) connections and/or transport layer security (TLS) orother cryptographically protected communication sessions), using one ormore computer networks or direct connections. However, it will beappreciated by those of ordinary skill in the art that such a systemcould operate in a system having fewer or a greater number of componentsthan are illustrated in FIG. 10 . Thus, the depiction of the system 1000in FIG. 10 should be taken as being illustrative in nature and notlimiting to the scope of the disclosure.

The various embodiments further can be implemented in a wide variety ofoperating environments, which in some cases can include one or more usercomputers, computing devices or processing devices which can be used tooperate any of a number of applications. In an embodiment, user orclient devices include any of a number of computers, such as desktop,laptop or tablet computers running a standard operating system, as wellas cellular (mobile), wireless and handheld devices running mobilesoftware and capable of supporting a number of networking and messagingprotocols and such a system also includes a number of workstationsrunning any of a variety of commercially-available operating systems andother known applications for purposes such as development and databasemanagement. In an embodiment, these devices also include otherelectronic devices, such as dummy terminals, thin-clients, gamingsystems and other devices capable of communicating via a network, andvirtual devices such as virtual machines, hypervisors, and other virtualdevices or non-virtual devices supporting virtualization capable ofcommunicating via a network.

In an embodiment, a system utilizes at least one network that would befamiliar to those skilled in the art for supporting communications usingany of a variety of commercially-available protocols, such asTransmission Control Protocol/Internet Protocol (“TCP/IP”), UserDatagram Protocol (“UDP”), protocols operating in various layers of theOpen System Interconnection (“OSI”) model, File Transfer Protocol(“FTP”), Universal Plug and Play (“UpnP”), Network File System (“NFS”),Common Internet File System (“CIFS”) and other protocols. The network,in an embodiment, is a local area network, a wide-area network, avirtual private network, the Internet, an intranet, an extranet, apublic switched telephone network, an infrared network, a wirelessnetwork, a satellite network, and any combination thereof. In anembodiment, a connection-oriented protocol is used to communicatebetween network endpoints such that the connection-oriented protocol(sometimes called a connection-based protocol) is capable oftransmitting data in an ordered stream. In an embodiment, aconnection-oriented protocol can be reliable or unreliable. For example,the TCP protocol is a reliable connection-oriented protocol.Asynchronous Transfer Mode (“ATM”) and Frame Relay are unreliableconnection-oriented protocols. Connection-oriented protocols are incontrast to packet-oriented protocols such as UDP that transmit packetswithout a guaranteed ordering.

In an embodiment, the system utilizes a web server that run one or moreof a variety of server or mid-tier applications, including HypertextTransfer Protocol (“HTTP”) servers, FTP servers, Common GatewayInterface (“CGI”) servers, data servers, Java servers, Apache servers,and business application servers. In an embodiment, the one or moreservers are also capable of executing programs or scripts in response torequests from user devices, such as by executing one or more webapplications that are implemented as one or more scripts or programswritten in any programming language, such as Java®, C, C# or C++, or anyscripting language, such as Ruby, PHP, Perl, Python or TCL, as well ascombinations thereof. In an embodiment, the one or more servers alsoinclude database servers, including without limitation thosecommercially available from Oracle®, Microsoft®, Sybase®, and IBM® aswell as open-source servers such as MySQL, Postgres, SQLite, MongoDB,and any other server capable of storing, retrieving, and accessingstructured or unstructured data. In an embodiment, a database serverincludes table-based servers, document-based servers, unstructuredservers, relational servers, non-relational servers, or combinations ofthese and/or other database servers.

In an embodiment, the system includes a variety of data stores and othermemory and storage media as discussed above which can reside in avariety of locations, such as on a storage medium local to (and/orresident in) one or more of the computers or remote from any or all ofthe computers across the network. In an embodiment, the informationresides in a storage-area network (“SAN”) familiar to those skilled inthe art and, similarly, any necessary files for performing the functionsattributed to the computers, servers or other network devices are storedlocally and/or remotely, as appropriate. In an embodiment where a systemincludes computerized devices, each such device can include hardwareelements that are electrically coupled via a bus, the elementsincluding, for example, at least one central processing unit (“CPU” or“processor”), at least one input device (e.g., a mouse, keyboard,controller, touch screen, or keypad), at least one output device (e.g.,a display device, printer, or speaker), at least one storage device suchas disk drives, optical storage devices, and solid-state storage devicessuch as random access memory (“RAM”) or read-only memory (“ROM”), aswell as removable media devices, memory cards, flash cards, etc., andvarious combinations thereof.

In an embodiment, such a device also includes a computer-readablestorage media reader, a communications device (e.g., a modem, a networkcard (wireless or wired), an infrared communication device, etc.), andworking memory as described above where the computer-readable storagemedia reader is connected with, or configured to receive, acomputer-readable storage medium, representing remote, local, fixed,and/or removable storage devices as well as storage media fortemporarily and/or more permanently containing, storing, transmitting,and retrieving computer-readable information. In an embodiment, thesystem and various devices also typically include a number of softwareapplications, modules, services, or other elements located within atleast one working memory device, including an operating system andapplication programs, such as a client application or web browser. In anembodiment, customized hardware is used and/or particular elements areimplemented in hardware, software (including portable software, such asapplets), or both. In an embodiment, connections to other computingdevices such as network input/output devices are employed.

In an embodiment, storage media and computer readable media forcontaining code, or portions of code, include any appropriate mediaknown or used in the art, including storage media and communicationmedia, such as, but not limited to, volatile and non-volatile, removableand non-removable media implemented in any method or technology forstorage and/or transmission of information such as computer readableinstructions, data structures, program modules or other data, includingRAM, ROM, Electrically Erasable Programmable Read-Only Memory(“EEPROM”), flash memory or other memory technology, Compact DiscRead-Only Memory (“CD-ROM”), digital versatile disk (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices or any other medium which canbe used to store the desired information and which can be accessed bythe system device. Based on the disclosure and teachings providedherein, a person of ordinary skill in the art will appreciate other waysand/or methods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit theinvention to the specific form or forms disclosed, but on the contrary,the intention is to cover all modifications, alternative constructions,and equivalents falling within the spirit and scope of the invention, asdefined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including,”and “containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected,” when unmodified and referring to physical connections, isto be construed as partly or wholly contained within, attached to, orjoined together, even if there is something intervening. Recitation ofranges of values herein are merely intended to serve as a shorthandmethod of referring individually to each separate value falling withinthe range, unless otherwise indicated herein and each separate value isincorporated into the specification as if it were individually recitedherein. The use of the term “set” (e.g., “a set of items”) or “subset”unless otherwise noted or contradicted by context, is to be construed asa nonempty collection comprising one or more members. Further, unlessotherwise noted or contradicted by context, the term “subset” of acorresponding set does not necessarily denote a proper subset of thecorresponding set, but the subset and the corresponding set may beequal.

Conjunctive language, such as phrases of the form “at least one of A, B,and C,” or “at least one of A, B and C,” (i.e., the same phrase with orwithout the Oxford comma) unless specifically stated otherwise orotherwise clearly contradicted by context, is otherwise understood withthe context as used in general to present that an item, term, etc., maybe either A or B or C, any nonempty subset of the set of A and B and C,or any set not contradicted by context or otherwise excluded thatcontains at least one A, at least one B, or at least one C. Forinstance, in the illustrative example of a set having three members, theconjunctive phrases “at least one of A, B, and C” and “at least one ofA, B and C” refer to any of the following sets: {A}, {B}, {C}, {A, B},{A, C}, {B, C}, {A, B, C}, and, if not contradicted explicitly or bycontext, any set having {A}, {B}, and/or {C} as a subset (e.g., setswith multiple “A”). Thus, such conjunctive language is not generallyintended to imply that certain embodiments require at least one of A, atleast one of B and at least one of C each to be present. Similarly,phrases such as “at least one of A, B, or C” and “at least one of A, Bor C” refer to the same as “at least one of A, B, and C” and “at leastone of A, B and C” refer to any of the following sets: {A}, {B}, {C},{A, B}, {A, C}, {B, C}, {A, B, C}, unless differing meaning isexplicitly stated or clear from context. In addition, unless otherwisenoted or contradicted by context, the term “plurality” indicates a stateof being plural (e.g., “a plurality of items” indicates multiple items).The number of items in a plurality is at least two, but can be more whenso indicated either explicitly or by context.

Operations of processes described herein can be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. In an embodiment, a process such as thoseprocesses described herein (or variations and/or combinations thereof)is performed under the control of one or more computer systemsconfigured with executable instructions and is implemented as code(e.g., executable instructions, one or more computer programs or one ormore applications) executing collectively on one or more processors, byhardware or combinations thereof. In an embodiment, the code is storedon a computer-readable storage medium, for example, in the form of acomputer program comprising a plurality of instructions executable byone or more processors. In an embodiment, a computer-readable storagemedium is a non-transitory computer-readable storage medium thatexcludes transitory signals (e.g., a propagating transient electric orelectromagnetic transmission) but includes non-transitory data storagecircuitry (e.g., buffers, cache, and queues) within transceivers oftransitory signals. In an embodiment, code (e.g., executable code orsource code) is stored on a set of one or more non-transitorycomputer-readable storage media having stored thereon executableinstructions that, when executed (i.e., as a result of being executed)by one or more processors of a computer system, cause the computersystem to perform operations described herein. The set of non-transitorycomputer-readable storage media, in an embodiment, comprises multiplenon-transitory computer-readable storage media and one or more ofindividual non-transitory storage media of the multiple non-transitorycomputer-readable storage media lack all of the code while the multiplenon-transitory computer-readable storage media collectively store all ofthe code. In an embodiment, the executable instructions are executedsuch that different instructions are executed by differentprocessors—for example, a non-transitory computer-readable storagemedium store instructions and a main CPU execute some of theinstructions while a graphics processor unit executes otherinstructions. In an embodiment, different components of a computersystem have separate processors and different processors executedifferent subsets of the instructions.

Accordingly, in an embodiment, computer systems are configured toimplement one or more services that singly or collectively performoperations of processes described herein and such computer systems areconfigured with applicable hardware and/or software that enable theperformance of the operations. Further, a computer system that implementan embodiment of the present disclosure is a single device and, inanother embodiment, is a distributed computer systems comprisingmultiple devices that operate differently such that the distributedcomputer system performs the operations described herein and such that asingle device does not perform all operations.

The use of any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate embodiments ofthe invention and does not pose a limitation on the scope of theinvention unless otherwise claimed. No language in the specificationshould be construed as indicating any non-claimed element as essentialto the practice of the invention.

Embodiments of this disclosure are described herein, including the bestmode known to the inventors for carrying out the invention. Variationsof those embodiments may become apparent to those of ordinary skill inthe art upon reading the foregoing description. The inventors expectskilled artisans to employ such variations as appropriate and theinventors intend for embodiments of the present disclosure to bepracticed otherwise than as specifically described herein. Accordingly,the scope of the present disclosure includes all modifications andequivalents of the subject matter recited in the claims appended heretoas permitted by applicable law. Moreover, any combination of theabove-described elements in all possible variations thereof isencompassed by the scope of the present disclosure unless otherwiseindicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

What is claimed is:
 1. A computer-implemented method, comprising:receiving a request to store a data object, the request specifying a setof attributes associated with the request; determining, based at leastin part on the set of attributes associated with the request, a set ofdata storage parameters, the set of data storage parameters specifying amovement classification of the data object amongst a set of data storagetier classifications, the set of data storage tier classificationscomprising at least a source data storage tier classification and adestination data storage tier classification; determining, based atleast in part on the set of data storage parameters, the source datastorage tier classification; classifying the data object ascorresponding to the source data storage tier; detecting a set ofchanges to the data object, the set of changes comprising a change in afrequency of access of the data object; determining, based at least inpart on the set of changes and the set of data storage parameters, thatthe data object is to be reclassified as corresponding to thedestination data storage tier classification; and re-classifying thedata object from the first directory source data storage tierclassification to the destination data storage tier classification. 2.The computer-implemented method of claim 1, wherein the source datastorage tier classification comprises a frequent access data storagetier classification and the destination data storage tier classificationcomprises one of an infrequent access data storage tier classificationor an archival data storage tier classification.
 3. Thecomputer-implemented method of claim 1, wherein the source data storagetier classification comprises a smart access data storage tierclassification and the destination data storage tier classificationcomprises a classification of one of a frequent access data storage tierclassification, an infrequent access data storage tier classification,or an archival data storage tier classification.
 4. Thecomputer-implemented method of claim 1, wherein the data objectcomprises a group of data objects associated with a logical datacontainer, and wherein detecting the set of changes to the data objectfurther comprises detecting the set of changes to the group of dataobjects associated with the logical data container.
 5. Thecomputer-implemented method of claim 4, wherein the set of changescomprises at least one of: a change in a frequency of access of at leastone data object in the group of data objects; or a size of the group ofdata objects.
 6. The computer-implemented method of claim 1, wherein theset of data storage parameters comprises a range of frequency of accessof the data object and a range of a size of the data object.
 7. Thecomputer-implemented method of claim 1, further comprising:re-classifying the data object as corresponding to a second destinationstorage tier classification based on detecting a second set of changesto the data object.
 8. A system, comprising: one or more processors; andmemory storing computer-executable instructions that, upon execution,cause the one or more processors to: create a job, specifying a set ofattributes, to store a group of data objects in a logical data containerassociated with a smart data tier classification, the set of attributescomprising a set of data storage parameters; classify the group of dataobjects as corresponding to a frequent access data storage tier among agroup of data storage tiers based on the set of data storage parameters,the group of data storage tiers comprising at least the frequent accessdata storage tier and an infrequent access data storage tier; detect aset of changes to at least one data object of the group of data objects,the set of changes comprising at least one of: a size of the group ofdata objects; or a modification time of at least one data object of thegroup of data objects; and reclassify the group of data objects from thefrequent access data storage tier to the infrequent access data storagetier.
 9. The system of claim 8, wherein the set of changes furthercomprises a change in a frequency of access of at least one data objectin the group of data objects.
 10. The system of claim 8, wherein the setof data storage parameters comprises at least one of a range offrequency of access of the data object and a range of a size of the dataobject.
 11. The system of claim 8, wherein the infrequent access datastorage tier comprises an archival data storage tier.
 12. The system ofclaim 8, wherein the smart data tier classification defines conditionsthat upon satisfaction, indicate movement of the group of data objectsbetween one or more of the group of data storage tiers.
 13. The systemof claim 8, wherein the computer-executable instructions further causethe one or more processors to re-classifying the data object ascorresponding to a different destination storage tier classificationbased on detecting a second set of changes to the data object.
 14. Anon-transitory computer-readable storage medium storing thereonexecutable instructions that, as a result of being executed by one ormore processors of a computer system, cause the computer system to atleast: create a job, specifying a set of attributes, to store a dataobject in a logical data container associated with a smart data storagetier, the set of attributes comprising a set of data storage parameters;associate the data object with a frequent access data storage tier amonga group of data storage tiers based on the set of data storageparameters, the group of data storage tiers comprising at least thefrequent access data storage tier and an infrequent access data storagetier; detect a set of changes to the data object, the set of changescomprising at least one of: a size of the data object; or a modificationtime of the data object; and reclassify the data object from thefrequent access data storage tier to the infrequent access data storagetier.
 15. The non-transitory computer-readable storage medium of claim14, wherein the set of changes further comprises a change in a frequencyof access of the data object.
 16. The non-transitory computer-readablestorage medium of claim 15, wherein the set of data storage parameterscomprises at least one of a range of frequency of access of the dataobject and a range of a size of the data object.
 17. The non-transitorycomputer-readable storage medium of claim 15, wherein the infrequentaccess data storage tier comprises one of the infrequent access datastorage tier or an archival data storage tier.
 18. The non-transitorycomputer-readable storage medium of claim 15, wherein the smart datatier classification defines conditions indicating placement of the groupof data objects into one of the group of data storage tiers.
 19. Thenon-transitory computer-readable storage medium of claim 15, wherein theexecutable instructions further cause the computer system tore-classifying the data object as corresponding to a differentdestination storage tier classification based on detecting a second setof changes to the data object.
 20. The non-transitory computer-readablestorage medium of claim 14, wherein the data object comprises a group ofdata objects associated with a logical data container, and wherein theexecutable instructions that cause the computer system to detect the setof changes to the data object further comprises executable instructionsthat cause the computer system to detect the set of changes to the groupof data objects associated with the logical data container.